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(57) Abstract: A narrow bandwidth broadcasting system for transmission of video and television via low bandwidth medium such 
as internet or radio links. 
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NARROW BANDWIDTH BROADCASTING SYSTEM 

The present invention relates to video and television systems and 
more particularly to a narrow bandwidth video delivery system. The 
5 system is particularly suitable for Internet broadcasting and is also suitable 
for transmission through radio links. 

Known television broadcasting systems require a wide bandwidth 
conduit for the transmission of the image data. This wide bandwidth is a 
10 problem which creates a bottleneck for live video transmission over 
delivery systems such as the telephone lines or radio links. 

Additionally, this large bandwidth requirement also limits the 
multiplication of broadcast channels and thereby limits viewer options, 

15 

It is an object of the present invention to overcome these 
disadvantages and to provide a television broadcast system which will 
require only a relatively narrow bandwidth. 

20 There are known systems which by transmitting poor quality image 

data can be transmitted over a relatively narrow bandwidth, such systems 
are used, for example, for remote security intruder detector systems and 
transmit a heavily degraded image which is sufficient to show movement 
of, for example, an intruder. These images are, however, not of 

25 sufficient quality to be acceptable for domestic television. 

The present invention in contrast, enables the transmission of a 
high quality image which will be accepted by viewers, as a television 
quality image. 

30 
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The present invention introduces a new method for narrow 
bandwidth broadcasting. The new method separates the video image into 
a foreground image and a background image and sends only the 
foreground image to the viewers. The system of the present invention 

5 utilises relates to any image separation techniques, and particularly, 
describes chroma- and depth-keying techniques. The broadcast system 
also captures the video camera identification, position, orientation and 
field of view size (position in x, y, z, zoom, tilt, roll and pan) for each 
video field and sends it together with the foreground image to the viewer. 

10 The broadcast system also includes an object tracking apparatus, which 
captures the position in space of objects that appear in the scene. In 
addition, the system comprises means to use several video cameras to 
capture the scene. 

15 The viewer side comprises a receiver and a processing unit that can 

generate a background image that will be composited with the transmitted 
foreground image. 

The background image can be either computer-generated locally at 
20 the viewer side, or in an alternative embodiment received from the 
broadcaster. 

The background image can be generated from a three dimensional 
model or a two dimensional image, pre loaded on the viewer's computer. 
25 The generated background image is enabled by the system to be in 
synchronization with the received camera parameters. By receiving the 
camera parameters for each video field and frame, the viewer's computer 
renders the graphical model and produces the appropriate background 
image, thus creating a realistic three-dimensional scene. 

30 
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The background image can also be constructed from a two 
dimensional image, pre loaded on the viewer's computer. The preloaded 
two dimensional image can be a higher resolution image that covers a 
wider view point range, much larger than needed for a single background 
5 image. By receiving the camera parameters, the viewer's computer 
selects the relevant portion of the original pre loaded image according to 
the pan, tilt, roll and zoom parameters and produces the appropriate 
background image. 

10 It is an object of the present invention to provide an economic 

broadcasting method which sends the viewer only pre-selected foreground 
objects from the scene. The rest of the image is generated by the viewer's 
system, which combines the pre-selected objects with the generated 
background. Considering the fact that foreground objects normally fill 

15 around 2 to 5% of the overall image area, the method of the present 
invention can, in a specific example, reduce a typical digital broadcast 
bandwidth from 4Mbit per second to 80Kbit per second, which is a typical 
modem bandwidth for telephone lines. This bandwidth size is also 
compatible with conventional radio station bandwidths. The presented 

20 method may be used to transmit television broadcasts via conventional 
radio transmission bandwidth or enable viewing real-time television shows 
via the Internet. Broadcasters can also use additional compression 
methods together with the present method, thus enabling additional 
reduction in the bandwidth. 

25 

The present invention can also comprise additional objects 
preloaded in the viewer's computer, to be added to the image as 
background or foreground objects. Such objects can be graphical 
animation, video clips or any images or graphics elements. 

30 
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By knowing the camera parameters, the foreground object position 
and the position of all other objects composed in the image, the present 
invention also enables the calculation of the x, y location of each object in 
the image. This enables the viewer to interact with the objects appearing 
5 in the image, by pointing or selecting an object in the image. When an 
object is pointed to the viewer's computer can identify which object was 
selected by using the x, y position, and perform any type of action related 
to the selected object. 

10 In a first embodiment the background at the receiver station is 

either a graphical model or a real background which has been generated 
or is actually present at the transmitter end. 

The present invention provides a narrow bandwidth broadcasting 
15 system including: 

a) video camera means for videoing of a complex scene with 
foreground and background objects. 

b) separation means for separating the video image into 
foreground and background images. 

20 c) position detecting means for providing the absolute position 

of each foreground object relative to the background or other fixed point 
in the each video field, 

d) camera parameter measurement means for measuring 
assigning camera parameters including :- 
25 i) absolute camera position relative to the background or 

a known fixed point, 

ii) camera setting for x, y, z, zoom, tilt, roll and pan, 

iii) all measurements for each video field, 
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e) first transmission means for transmission of a graphical 
model or real image that will be used to generate the background image 

on a receiver site. 

f) second means for transmission of the video image of each 

5 foreground object, 

g) third transmission means for transmission of the absolute 

position of the object in each video frame, 

h) fourth transmission means for transmission of the camera 

parameters for each video frame, 
10 i) receiver means including first storage means for storage of 

the background graphical model or real image at the receiver site in a 
suitable storage, 

j) second storage means for storage of video images of each 
foreground object, 

15 k) third storage means for storage of the position of each 

foreground object, 

1) fourth storage means for storage of camera position and 

parameters, 

m) first processor means for reconstruction of the background 
20 image as should be seen by the camera using the stored background 
graphical model or real image and the camera parameters at the receiver 
site, 

n) second processor means for addition of foreground to 
background, and 

25 o) display means for display of combined background and 

foreground image. 

In a second embodiment the background is not transmitted but is a 
known graphical model or a known image which is stored at the receiver 
30 end. 
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The present invention also provides a narrow bandwidth 
broadcasting system including: 

a) video camera means for videoing of a complex scene with 
foreground and background objects, 
5 b) separation means for separating the video image into 

foreground and background images, 

c) position detecting means for providing the absolute position 
of each foreground object relative to the background or other fixed point 
in the each video field, 
iq d) camera parameter measurement means for measuring camera 

parameters including:- 

i) absolute camera position relative to the background or 
a known fixed point, 

ii) camera settings for x, y, z, zoom, tilt and pan, 
15 iii) all measurements for each video field, 

e) first transmission means for transmission of the video image 
of each foreground object, 

f) second transmission means for transmission of the absolute 
position of the object in each video frame, 

20 g) third transmission means for transmission of the camera 

parameters for each video frame, 

h) receiver means including: - 

first storage means for storage of video images of each 
foreground object, 

25 i) second storage means for storage of the position of each 

foreground object, 

j) third storage means for storage of camera position and 

parameters, 

k) fourth storage means for storage of a known image or 
30 graphical model, 
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1) background generating means for generating background, 
m) first processor means for reconstruction of the background 
image as should be seen by the camera using the stored background 
graphical model or a known image and the camera parameters at the 
5 receiver site, 

n) second processor means for addition of the foreground 
objects to the background, and 

o) display means for display of the combined background and 
the foreground image to display said foreground objects in a correct 
10 positional relationship to said background. 

The present invention also provides in a preferred embodiment 
character generating means for inserting additional foreground objects at 
the viewers' site. 

15 

In a further preferred embodiment the first transmission means 
comprises means for transmitting a difference graphical model 
corresponding to differences in a background image between a first video 
frame and a second video frame. 

20 

Preferably said first storage image means comprises means for 
storing a difference graphical model and in which said first processor 
means also comprises means for constructing a modified graphical model 
by combining said previous stored graphical model with said difference 
25 graphical model. 

In a further preferred embodiment the first transmission means 
comprises means for transmitting a difference image corresponding to 
differences in a background image between a first video frame and a 
30 second video frame. 

7 
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Preferably said first storage image comprises means for storing 
difference image data and in which said first processor means also 
comprises means for constructing a changed background image by 
5 combining said stored background image with said difference image data. 

Preferably said foreground image data is transmitted in an RGB or 
other standard format. 

to In a first embodiment there is provided chroma key separation 

means based on the colour difference between the background and 
foreground images. 

In a second embodiment there is provided depth measurement 
15 means for measuring the depth of each pixel of both background and 
foreground images. 

Preferably said apparatus comprises transmission means for 
transmitting said chroma key separation data relating to the background 
20 and foreground images. 

Alternatively said apparatus comprises transmission means for 
transmitting said pixel chroma key separation data or said depth 
measurement data. 

25 

Preferably said receiver means includes fifth storage means for 
storing said depth measurement data of the background image. 

Preferably said second processor means includes means for 
30 combining said foreground and background image data transmitted on an 
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RGB or other standard format with said chroma key separation data or 
said depth measurement data. 



The present invention provides, for the embodiment in which a 
5 graphical model or real image is transmitted, a method of narrow 
bandwidth broadcasting comprising the steps of: 

a) videoing of a complex scene with foreground and 
background objects, 

b) separating the video image into foreground and background 

10 images, 

c) detecting the absolute position of object relative to the 
background or other fixed point in the each video field, 

d) measuring the parameters of a video camera including:- 

i) the absolute camera position relative to the 
15 background or a known fixed point 

ii) camera settings for x, y, z, zoom, tilt and pan 

iii) all measurements for each video field 

e) transmitting a graphical model or a real image, that will be 
used to generate the background image on the receiver site, 

20 f) transmitting the video image of each foreground object, 

g) transmitting the absolute position of the object in each video 
frame, 

h) transmitting the camera parameters for each video frame, 

i) storing the background graphical model or image at a 
25 receiver site in a suitable storage, 

j) storing the video images of each foreground object, 
k) storing the position of each foreground object, 
1) storing the camera position and parameters, 
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m) reconstructing the background image as should be seen by 
the camera using the stored background graphical model or image 
and the camera parameters at the receiver site, 
n) adding the foreground to the background in a processor, 
5 o) displaying a combined image of foreground and background. 

For the embodiment in which a graphical model or known 
image is stred at the receiver site, the present invention also 
provides a method of narrow bandwidth broadcasting comprising 

10 the steps of: 

a) videoing of a complex scene with foreground and 

background objects, 

b) separating the video image into foreground and background 

images, 

15 C ) detecting the absolute position of object relative to the 

background or other fixed point in the each video field, 

d) measuring the parameters of a video camera including:- 

i) the absolute camera position relative to die 
background or a known fixed point 
20 ii) camera settings for x, y, z, zoom, tilt and pan 

iii) all measurements for each video field 

e) transmitting the video image of each foreground object, 

f) transmitting the absolute position of the object in each video 
frame, 

25 g) transmitting the camera parameters for each video frame, 

h) storing a known image or graphical model at a receiver site, 

i) storing the video images of each foreground object, 
j) storing the position of each foreground object, 

k) storing the camera position and parameters, 
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1) grabbing a known image or graphical model from known 

collection of images or models, 
m) reconstructing the background image as should be seen by 

the camera using the stored background graphical model or 
5 image and the camera parameters at the receiver site, 

n) adding the foreground object images to the generated 

background in a processor, 
o) displaying on a VDU display a combined image of 

foreground and background. 

10 

In addition, foreground objects generated at the receiver site end 
could be added to the scene if required. 

Embodiments of the present invention will now be described, by 
15 way of example with reference to the accompanying drawings in which :- 
Figure 1 shows schematically a video screen including background 
and foreground objects, 

Figure 2 shows in block diagram form a transmitter system for the 
present invention, and 
20 Figure 3 shows in block diagram form a receiver system for the 

present invention. 

With reference now to Figure 1, the video scene 10 comprises a 
background 12 and foreground objects 14. As shown the background is a 
25 real background, where foreground and background separation will be 
performed by means like chroma keying or depth keying. 

The foreground objects 14 could be fixed in position as in the case 
of the box 141 or could move about as in the case of the person 142. 

30 

11 



WO 01/13645 PCT/GBOO/03174 

A video camera 16 is positioned to video the scene. The video 
camera 16 may be provided with position sensing means 160 which may 
be scanned, for example, by detectors 162, 164 to give the exact position 
of the camera 16 in three dimensions x, y, z. 

5 

The camera 16 may also be equipped with pan, roll, zoom, tilt 
sensors 166 which, in conjunction with the x, y, z measurements, will 
ascertain the exact camera parameters relative to the background to a 
fixed point, e.g. P on the floor. The position of the background relative 
to the camera will therefore be known in each video field. 



10 



15 



The system also requires to know the position in each video frame 
of each foreground object. This can be done by sensors, e.g. 1410 on 
each object or by image processing. 

The alternative ways of operating this system will now be 
described. 

In order to separate the video scene into foreground and 
20 background images, several techniques such as chroma keying, depth 
segmentation and objects' edge-detection can be used. 

Chroma keying is the most common technique used in virtual 
studios for live television and video productions. The foreground objects 
25 are presented in front of a chromakey panel background. The color of the 
chroma key panel is then detected and replaced by a virtual background. 
This replacement is done automatically using dedicated hardware and 
software and enables outputting a combined video signal of both the real 
foreground objects and the virtual background to be transmitted. There is 
30 no limitation regarding the color of the chromakey panel, although blue is 

12 
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the preferred choice as it is farthest from normal white flesh tones. When 
black foreground objects or actors are used, green is a preferred choice. 



One of the main problems with chroma keying in virtual studios is 
5 in cases where the objects or actors are of similar color to the chromakey 
panel. In that scenario it is possible to see keying mistakes due to 
replacing part of the foreground objects or actors with the background 
image data. Because of these keying problems it is recommended that die 
color of the actor's clothing be different from the chromakey panel color. 

10 

Depth keying is the preferred choice in real studios, when it is hard 
to separate the foreground object from the background by image 
processing means. Depth segmentation can be done by using various 
techniques: 

15 

A first such technique is to generate a light or a sound pulse that is 
projected toward the targeted scene. The pulse is reflected and the 
reflected pulse is received at the detection device, where the time of flight 
and intensity of the reflected pulse are measured. Taking into 

20 consideration both the intensity and the time of flight, it is possible to 
measure the distance of every point on an object surface from the 
detecting device. In a preferred configuration the detecting device is 
combined with a video camera in such a way that for every pixel in the 
video image, the detection device measures the intensity and the time of 

25 flight from the corresponding position in space, for example WO 
97/12326 and WO 97/01 111. 

Another method is triangulation. By using several images of the scene 
from various angles, it is possible to calculate the depth of a point in the 
30 scene by knowing the position from which each image is shot, the camera 

13 
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parameters for each image shot which enables calculation of the relative 
placement of the point between the different images. 

Other known techniques used are alternate defocusing and phase 
detection but these will not be described in further detail. 

A further separation method is edge detection, which can be 
performed as follows using one of several techniques: 

a. Backfiround subtraction . This is based on capturing a reference 
image, which does not contain the preferred objects. By using reference 
image subtraction it is possible to detect the object edges. 

b. Texture separation . This technique is based on the different texture 
of the preferred object from the background texture. 

c. Color separation . This technique is based on the different color of 
the preferred objects from the background color. 

To enable the viewers system to assimilate the processed background 
with the received foreground object, the viewers system must also receive 
information on the camera parameters for each video field. Tracking of 
camera position, orientation and field of view size (position in x,y,z, 
zoom, tilt, roll, pan) is a common technique used especially in virtual 
studios or electronic advertising in sports. Knowing the camera position, 
orientation and field of view size enables the performance of several 
actions automatically such as: replacing a chroma key billboard in a three 
dimensional scene, tracking static objects and combining an additional 
foreground image with a background image based on a real image or a 
computer generated image into a combined image keeping the right 
perspective between the foreground and the background parts. 
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There are three main techniques to capture the camera position, 
orientation and field of view size. The first is based on electro-mechanical 
or electro-optical sensors which are located on the camera and measure 
the rotation axes (tilt, roll and pan) as well as the status of the zoom and 

5 focus engine. The second is based on image processing of the video 
sequence, which can be done by pattern recognition of a visible pattern in 
the image or by calculating the relative correlation from frame to frame. 
The third technique is tracking the motion of markers placed on the 
camera. By knowing the camera position, orientation and field of view 

10 size it is possible to automatically find the exact position of any object in 
the video image at any given time using initial positioning data at a certain 
time, regardless of any change of the camera position, orientation and 
field of view size (position in x,y,z, zoom, pan, tilt, roll) during the video 
film sequence. 

15 

In order to combine additional three-dimensional objects in the' 
viewers video scene, information regarding the position of the received 
foreground object is required. By knowing the foreground objects 
position, it is possible, for example, to realistically combine objects in 
20 front of the foreground object. The position of the foreground object for 
each video field must be captured by the broadcaster and sent with the 
transmitted video signal to the viewer. 

Two main techniques to track objects in a scene are described. The 
25 first is by image processing and the second is by tracking sensors, 
markers, receivers or any marking tags, by both optical and electronic 
means. 

In a preferred embodiment of the present invention, the broadcast 
30 system includes a virtual studio system. A video camera is used to capture 

15 
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both the objects and the blue or other suitable colour as hereinbefore 
described screen. For each video frame the image is then transferred to a 
chroma keyer, which separates the foreground objects from the blue 
background. The resulting image contains the objects and a uniform color 

5 background. The foreground objects are then keyed out and transmitted 
to the viewer together with camera parameters and the objects tracking 
information. The viewers' system contains a receiver and a processing 
unit. By using the camera and the objects tracking information, the 
viewers system can merge the foreground object, such as an actor, with an 

10 artificial 3-dimensional background edited by computer to create a 
realistic, 3-dimensional scene. 

Apparatus suitable for the system is described with reference to 
figures 2 and 3. Figure 2 shows processing and transmitter circuitry 200 
15 in block diagram form. 

If the scene includes a real background then in Figure 2 the video 
camera 16 provides a combined output of the scene. This includes both 
background and any foreground objects. If the scene is a chroma key 

20 (e.g. blue) background then the camera 16 is used to capture the 
foreground image. The background may then be added either by normal 
chroma key techniques or a 3D model may be used as described 
hereinafter. A preferred use of this invention is with a 3D graphical 
model. If a real background image is used then its size must be equal to 

25 or preferably greater than the video image size. A background of equal 
size is only useful in cases where the camera does not move. The 
background image for each video field is appropriately extracted from the 
high resolution image, according to the camera parameters. 
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Unlike the 3D model, the real background image is only of use 
with a fixed position camera thus representing a special case. 



When the 3D model is used the camera can move since the camera 
5 position is known and the 3D model is therefore adjustable to the new 
camera position. A plurality of 3D models can be stored in a store 211 
and transmitted to the receiver for use with the foreground object and/or 
to provide an interactive display. 

io The composite video image may be stored in a temporary store 202 

which serves to buffer the image for further processing in a 
foreground/background separation processor 204. This processor 204 
receives pixel data from a separation detection unit 206 which may also be 
stored in a buffer store 208. This data enables the separation of the 

15 composite image data on a pixel by pixel basis into background and 
foreground image data. 

As stated above, the background can be obtained by videoing the 
scene without a foreground object to provide a high resolution background 
20 image. This is stored in background store 210 for subsequent 
transmission. Alternatively, backgrounds comprising 3D models are 
stored in a further store 211 and one of these can be selected for 
transmission to the receiver site at which the viewer is present. 

25 The background pixel data is stored in a store 210 and the 

foreground object data in a store 212. 

The apparatus further comprises a camera position detector circuit 
214 which may receive data from, for example, the detectors 162 and 
30 164, to provide an exact x, y, z position of the camera for each video 
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frame. All of the circuitry is preferably synchronised to the main studio 
signal to ensure that all data for each video frame is synchronised. 



The camera position data is stored in a suitable temporary store 

5 216. 

The camera parameters (tilt, pan, roll, zoom) are detected for each 
frame in a detector circuit 218 which may be connected, for example, to 
receive the output of detector 166 in Figure 1. These parameters, again 
io synchronised to the main studio signal, are stored in a store 220. 

The position of each foreground object is detected in a detector 
circuit 222 which may be of the type 1410 as shown in Figure 1. 

15 The foreground object positions are stored in a store 224 for each 

video frame. 

Stores 216, 220, 224 may, as shown, be connected to respective 
transmit circuits 226, 228, 230 or these could be combined, as indicated 
20 by the dotted lines, into a single transmit circuit 232. 

In operation the background stored in store 210 will be formulated 
to be transmitted by a transmit circuit 234 and then transmitted by a 
suitable conversion/transmitter circuit 236 over a broadcast media 238. 
25 The background may in a preferred embodiment be transmitted for 
example prior to transmission of the data concerning foreground objects. 
The background may be transmitted over a relatively long period, e.g. 
video frames to be received and stored at the receiver site prior to 
transmission of the data concerning foreground objects. The background 
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store at the receiver site (to be described hereinafter) will therefore hav 
either a graphical model or a real background image stroed therein. 



Alternatively, the selected 3D graphical model from store 211 will 
5 be transmitted. 

Each foreground object, the data for which is stored in store 212, is 
transmitted via conversion/transmitter circuit 240 to be transmitted on 
media 238 by transmitter 236. The transmission will be synchronised 
10 preferably to the studio sync. 

Preferably following this transmission the camera position, camera 
parameters and foreground object position from stores 226, 228, 230 will 
then be transmitted for each frame. This will also be similarly 
15 synchronised. 

During the next and subsequent frames only the foreground object 
and camera position, camera parameters and foreground object positions 
will be transmitted. The background is assumed in this example to be 
20 constant. 

In the event that the background can vary, for example, by 
movement of a background object, a difference background signal can be 
generated to accommodate this change. This signal is generated by a 

25 comparison circuit 242 which compares the present with the previous 
background on a frame by frame basis. The camera position and 
parameter data is also used to determine the background difference since 
the background will vary for 3D models as the camera position and 
parameters vary and therefore the data is used to control the comparison 

30 circuitry. Small variations are detected and stored in a difference store 
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244 and these can be transmitted again suitably coded for interpretation by 
the receiver. 



A suitable exemplary receiver circuitry 300 is shown in Figure 3. 

5 

The narrow bandwidth transmission may be received either by a 
radio aerial 302 or telephone line connection 304 and also by any normal 
broadcast method such as cables, satellite and aerial TV broadcast. A 
suitable modem/buffer circuit 306 will decode and temporarily store as 
10 necessary any incoming data, the identity of which will have been suitably 
coded (e.g. by a header) to enable it to be identified. 

The receiver circuitry will need to be in sign synchronism and will 
preferably obtain sync information from the incoming signal and generate 
15 sync timing signals in sync/timer circuit 308. These are symbolically 
shown as outputs 309 and are in known manner connected to synchronise 
all circuits in the receiver. 

Initially the transmitted background data will be received and 
20 stored in a background image store 310. This background is then used 
continuously unless updated by a difference signal suitably coded. The 
difference signal data may be stored in a separate difference store 313 and 
used to update the background data in store 310, in a processor 311 which 
will also receive camera parameters to generate the correct background. 

25 

For each frame thereafter the foreground object, camera position, 
camera parameter and foreground object position data will be received in 
buffer 306 and since it is suitably coded, it will be sorted and stored in 
respective stores 312, 314, 316 and 318. 

30 
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The store 310 can also be used to store a 3D graphical model which 
can be input at the receiver site either directly or as explained with 
reference to figure 2 via the transmission medium from the transmitter of 
figure 2. 

5 

The foreground object data is then combined with the positional 
data in a processor 320 and the complex output of this is input into a 
combiner processor circuit 322 in which the foreground object is correctly 
positioned with respect to and combined with the background. The 
10 processors 320 and 322 may possibly be combined in a single processor. 
The output of processor 322 is then displayed on a TV/VDU display 324. 

If the background image changes then circuits 242, 244 in the 
transmission circuit will transmit a difference signal which will be coded 
15 as such. The background image store 310 will be updated to provide the 
new background. This may be necessary, for example, if the background 
is 3D and the camera moves. Any such changes per frame will be very 
small, requiring limited bandwidth. 

20 The system of the present invention can therefore accommodate 

large movements in foreground objects and, if required, changes in the 
background image. 

In a second embodiment the background image is created at the 
25 receiver using a suitable video generator 326. The background could be 
generated by the viewer using a suitable computer or could be selected 
from, for example, a plurality of backgrounds stored in an archive store 
328. The background can be selected to conform to a known virtual 3D 
background which could be used in the studio to thereby conform the 
30 movements in the studio to those at the viewers 1 site. The background 
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could be identified to the viewer by a simple code, e.g. a number of 
letters. 



The receiver may also include a pointing device 321 which can 
5 select a position on the VDU 324 under the control of a controller 323 
which is preferably manually operated by a viewer. The pointing device 
321 can, in combination with the control 323 and an object 
information/storage device 325, provide information relating to an object 
on the VDU 324 in the selected position. The information can be stored 
10 in the store 325 from a local source, for example, a video disc player 
3250 or it could be obtained from the foreground object store 318 having 
been transmitted from the transmitter of figure 2. 
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CLAIMS 

L A narrow bandwidth broadcasting system including: 
5 a) video camera means for videoing of a complex scene with 

foreground and background objects 

b) separation means for separating the video image into 
foreground and background images, 

c) position detecting means for providing the absolute position 
10 of each foreground object relative to the background or other fixed point 

in each video field, 

d) camera parameter measurement means for measuring 
assigning camera parameters including :- 

i) absolute camera position relative to the background or 
15 a known fixed point, 

ii) camera setting for x, y, z, zoom, tilt, roll and pan, 

iii) all measurements for each video field, 

e) first transmission means for transmission of a graphical 
model or real image that will be used to generate the background image 

20 on a receiver site, 

f) second means for transmission of the video image of each 

foreground object, 

g) third transmission means for transmission of the absolute 
position of the object in each video frame, 

25 h) fourth transmission means for transmission of the camera 

parameters for each video frame, 

i) receiver means including first storage means for storage of 
the background graphical model or real image at the receiver site in a 
suitable storage, 
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j) second storage means for storage of video images of each 
foreground object, 

k) third storage means for storage of the position of each 
foreground object, 

5 1) fourth storage means for storage of camera position and 

parameters, 

m) first processor means for reconstruction of the background 
image as should be seen by the camera using the stored background 
graphical model or real image and the camera parameters at the receiver 
io site, 

n) second processor means for addition of foreground to 
background, and 

o) display means for display of combined background and 
foreground image. 

15 

2. A narrow bandwidth broadcasting system including: 

a) video camera means for videoing of a complex scene with 
foreground and background objects, 

b) separation means for separating the video image into 
20 foreground and background images, 

c) position detecting means for providing the absolute position 
of each foreground object relative to the background or other fixed point 
in each video field, 

d) camera parameter measurement means for measuring 
25 assigning camera parameters including :- 

i) absolute camera position relative to the background or 
a known fixed point, 

ii) camera settings for x, y, z, zoom, tilt, roll, and pan, 

iii) all measurements for each video field, 
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e) first transmission means for transmission of the video image 
of each foreground object, 

f) second transmission means for transmission of the absolute 
position of the object in each video frame, 

5 g) third transmission means for transmission of the camera 

parameters for each video frame, 

h) receiver means including :- 

first storage means for storage of video images of each 
foreground object, 

10 i) second storage means for storage of the position of each 

foreground object, 

j) third storage means for storage of camera position and 
parameters, 

k) fourth storage means for storage of a known image or 
15 graphical model, 

1) background generating means for generating background, 
m) first processor means for reconstruction of the background 
image as should be seen by the camera using the stored background 
graphical model or a known image and the camera parameters at the 
20 receiver site, 

n) second processor means for addition of the foreground 
objects to the background; and 

o) display means for display of the combined background and 
the foreground image to display said foreground objects in a correct 
25 positional relationship to said background. 

3. A narrow bandwidth broadcasting system as claimed in claim 1 or 

claim 2 including the present invention also provides in a preferred 

embodiment character generating means for inserting additional 

30 foreground objects at the viewers' site. 
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4. A narrow bandwidth broadcasting system as claimed in claim 1, in 
which the first transmission means comprises means for transmitting a 
difference graphical model corresponding to differences in a background 

5 image between a first video frame and a second video frame. 

5. A narrow bandwidth broadcasting system as claimed in claim 4 in 
which the first storage image means comprises means for storing a 
difference graphical model and in which the first processor means also 

io comprises means for constructing a modified graphical model by 
combining said previous stored graphical model with second difference 
graphical model. 

6. A narrow bandwidth broadcasting system as claimed in claim 1 in 
15 which the first transmission means comprises means for transmitting a 

difference image corresponding to differences in a background image 
between a first video frame and a second video frame. 

7. A narrow bandwidth broadcasting system as claimed in claim 1 or 
20 claim 6 in which said first storage image comprises means for storing 

difference image data and in which said first processor means also 
comprises means for constructing a changed background image by 
combining said stored background image with said difference image data. 

25 8. A narrow bandwidth broadcasting system as claimed in any one of 
claims 1 to 7 in which said foreground image data is transmitted in an 
RGB or other standard format. 

9. A narrow bandwidth broadcasting system as claimed in any one of 
30 claims 1 to 8 in which there is provided chroma key separation means 
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based on the colour difference between the background and foreground 
images videoed by the video camera in order to determine the separation 
of the foreground and the background. 

10. A narrow bandwidth broadcasting system as claimed in any one of 
claims 1 to 8 in which there is provided depth measurement means for 
measuring the depth of each pixel of both background and foreground 
image videoed by said video camera. 

11. A narrow bandwidth broadcasting system as claimed in claim 9 or 
claim 10 in which said apparatus comprises fifth transmission means for 
transmitting said chroma key separation data or said pixel depth 
measurement data. 



12. A narrow bandwidth broadcasting system as claimed in claim 9, 10 
or claim 11 in which said receiver means includes fifth storage means for 
storing said chroma key separation data or said depth measurement data. 

13. A narrow bandwidth broadcasting system as claimed in claim 12 in 
which said second processor means includes means for combining said 
foreground and background image data transmitted on an RGB or other 
standard format with said chroma key separation or said depth 
measurement data. 

14. A method of narrow bandwidth broadcasting comprising the steps 
of: 

a) videoing of a complex scene with foreground and 
background objects, 

b) separating the video image into foreground background 

image, 
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c) detecting the absolute position of object relative to the 
background or other fixed point in the each video field, 

d) measuring the parameters of a video camera including:- 

i) the absolute camera position relative to the 
5 background or a known fixed point 

ii) camera settings for x, y, z, zoom, roll, tilt and pan 

iii) all measurements for each video field 

e) transmitting a graphical model or real image that will be 
used to generate the background image on the receiver site, 

10 f) transmitting the video image of each foreground object, 

g) transmitting the absolute position of the object in each video 

frame, 

h) transmitting the camera parameters for each video frame, 

i) storing the background graphical model or image at a 
15 receiver site in a suitable storage, 

j) storing the video images of each foreground object, 
k) storing the position of each foreground object, 
1) storing the camera position and parameters, 
m) reconstructing the background image as should be seen by 
20 the camera using the stored background graphical model or image and the 
camera parameters at the receiver site, 

n) adding the foreground to the background in a processor, 
o) displaying a combined image of foreground and background. 

25 15. A method of narrow bandwidth broadcasting comprising the steps 
of: 

a) videoing of a complex scene with foreground and 
background objects, 

b) separating the video image into foreground background 
30 images, 
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c) detecting the absolute position of object relative to the 
background or other fixed point in the each video field, 

d) measuring the parameters of a video camera including:- 

i) the absolute camera position relative to the 
5 background or a known fixed point 

ii) camera settings for x, y, z, zoom, tilt and pan 

iii) all measurements for each video field 

e) transmitting the video image of each foreground object, 

f) transmitting the absolute position of the object in each video 
10 frame, 

g) transmitting the camera parameters for each video frame, 

h) storing a known image or graphical model at the receiver 
site to comprise one of known collection of images or models, 

i) storing the video images of each foreground object, 
15 j) storing the position of each foreground object, 

k) storing the camera position and parameters, 
1) grabbing a known image or graphical model from known 
collection of images or models, 

m) reconstructing the background image as should be seen by 
20 the camera using the stored background graphical model or image and the 
camera parameters at the receiver site, 

n) adding the foreground object images to the generated 
background in a processor, 

o) displaying on a VDU display a combined image of 
25 foreground and background. 

16. A method of narrow bandwidth broadcasting as claimed in claim 15 
in which said transmitting steps e) to g) are by telephone radio, cable 
satellite or aerial TV transmission systems. 

30 
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17. A method of naiTow bandwidth transmission as claimed in claim 15 
in which said transmitting steps e) to g) are by Internet transmission 
media. 

5 18. A method of narrow bandwidth transmission as claimed in any one 
of claims 15 to 17 in which the background image is a rendered from a 3 
dimensional model. 

19. A method of narrow bandwidth broadcasting as claimed in any one 
10 of claims 15 to 18 in which the background is calculated by selecting a 

portion from a high resolution image that represents a wider point of view 
range. 

20. A method of narrow bandwidth broadcasting as claimed in any one 
15 of claims 15 to 19 in which the system comprises an interactive system by 

use of the knowledge of the camera parameters, the position of the object 
and the background image said method comprising the steps of using a 
pointing device to identify an object and including means for providing 
additional information relating to such identified object. 

20 
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(57) Abstract: A narrow bandwidth broadcasting system for transmission of video and television via low bandwidth medium such 
as internet or radio links. To save bandwith, video scenes are separated into foreground and background objects and only foreground 
objects are sent to the viewers. However, graphical model or real image may be sent as a reference background and updated according 
to its own movement. The broadcast system also captures the video camera identification, position, orientation and field of view 
size for each video fields and sends it together with the foreground object to the viewer. The broadcast system also includes an 
object tracking apparatus, which captures the position in space of objects that appear in the scene. Shuch parameters allow the 
implementation of a 3-dimensional rendering at the receiver side. 
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